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Summary;  It  is  shown  that  functional  equations 
of  the  form  f(p)  -  Max  TJc(f(p))  which  occur  in 

the  theory  of  optimal  inventory  and  optimal 
allocation  may  be  Bolved  by  very  simple  iterative 
processes  under  certain  conditions. 


ON  THE  COMPUTATIONAL  SOLUTION  OF  SOME  FUNCTIONAL  EQUATIONS 
IN  THE  THEORY  OF  DYNAMIC  PROGRAMMING 

Richard  Bellman 


$1 .  Introduction 

The  theory  of  dynamic  programming  gives  rise  to  a  general 
class  of  functional  equations  having  the  form 

f(p)  -  Ma-  Tk(f(p))  (1.1) 

k  K 

where  Tk  is  a  transformation.  In  general,  these  equations  cannot 
be  resolved  explicitly,  and  iterative  techniques  must  be  used  to 
determine  the  solution. 

Since  the  numerical  evaluation  of  the  maximum  over  k  is, 
in  general,  quite  onerous,  it  is  desirable  to  have  an  alternative 
method  of  computation  of  successive  approximation  to  the  solutions. 

We  shall  Bhow  how  an  alternative  method  can  be  obtained  for 
two  important  classes  of  functional  equations,  those  occurring 
in  multi-stage  allocation  processes,  an  example  of  which  is 


f (x )  -  Max  Qg(y)  +  h(x-y)  +  f(ay  +  b(x-y))^ 
0<y<x 


(1.2) 
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and  those  occurring  in  optimal  Inventory  theory,  an  example  of 
which  is 


f(x)  -  Min  fv(x,y)  +  *  Q  f  (0)  (l-h(y) )  +  £  f  (y-e)h'  (s)ds  ]  1 
y^x  l-  o  -J 


(1.3) 


^2 .  Optimal  Allocation 

It  is  known  (see  [l]  ,  [^}  )  that  if 

a.  0  <  a,  b  <  1 

b.  g(0)  -  h(O)  -  0 

c.  g(x)  and  h(x)  are  continuous  in  Q),cJ 


(2.1) 


then  the  unique  continuous  solution  to  (1.2)  in  0?,<Q  which  is 
zero  at  x  •  0  la  obtained  by  starting  with  an  arbitrary  continu-e 
ous  function  fQ(x)  zero  at  x  -  0,  and  iterating 


fi  -  T(f0) 
fa  -  T  ( f  1 ) 


(2.2) 


and  so  on,  with  f(x)  -  lirn  fR. 

u - >  00 

Let  us  now  assume  that  in  (1.2)  the  maximum  is  taken  inside 
the  interval  (0*x3  f°r  aH  x  >  0,  and,  indeed,  for  each  approxi¬ 
mation  in  (2.2)  above. 

Then  we  have 
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fnx,(*)  -  s(yn)  +  h(x-yn)  +  fn(ayn  +  b(x-yn)) 


(2.3) 


where  yn  -  yn(x)  Is  determined  by  the  equation 


0  -  g!(y)  -  h'(x-y)  +  (a-b)f^(ay  +  b(x-y)) 


(2.4) 


Returning  to  (2.3),  we  obtain 


uy 

fn+i(x)  •  &’(yn^  -  h'(’t-yn)  +  (ft-to)fi(ayn  +  b(x-yn)0  S5r 


+  h'(x-yn)  +  bf^(ayn+  b(x-yn)) 
-  h'(x-yn)  +  f„(ay n  +  b(x-yn)) 


(2-5) 


From  (2.4)  we  see  that  f^(x)  determines  yQ(x),  while  (2.5) 
shows  that  yQ(x)  and  fg(x)  determine  f{(x),  and  so  dn,  recurrently. 
From  this  it  is  clear  that  the  important  sequence  is  actually 
fr;<x)}  ,  which  may  now  be  computed  recurrently  in  very  simple 
fashion. 

Actually,  we  obain  more.  If  fQ(x)  agrees  with  the  actual 


solution  f(x)  in  same  Interval  U>,c]  »  fi  (x)  will  agree  in  a  larger 
interval,  f«(x)  in  a  still  larger  interval,  and  so  on.  Thus 
the  actual  solution  may  be  obtained  by  recurrence. 


Optimal  Inventory 

It  is  known  (aee  Q3  »  DO)  that  if 


§ 
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a.  0  <  *  <  1 


b.  h»(s)  £  0 
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(s)ds  **  1 


(3-1) 


c.  v(w)  Is  continuous  in  any  finite  interval  0  <  y  <  c, 

the  solution  of  (1.3)  may  be  obtained  by  iteration  using  the 
sequence 


f„+l  -  T(fn> 


(3.2) 


Let  us  assume  again  that  the  minimum  is  always  taken  inside. 


Then 


fn+1(x)  -  V(x 


nn 

,yn)  +  ;\£fn(o)(i-h(yn))  *  J  fn(yn-s)h'(s)d83 


(3-3) 


where  y^  is  determined  by  the  equation 

J$2L-  (x,yn)  +  /  t'n  (y-s)h'  (s)ds  -  0 


(3.4) 


Returning  to  (3*3 )»  w®  obtain 


Ci<*>  -  ir-  (x»yn) 


+  £g«(yn-x)  +  *  Jn  fi(yn~0)h'  ^8)d8U 


(5-5) 
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This  again  furnishes  a  very  simple  computational  algorithm 
for  determining  y0  given  f^  and  then  f{  given  yQ  and  f'0,  and  so  on. 

,£4.  How  to  Obtain  Internal  Extrema 

To  insure  that  the  minimum  in  (3.2)  is  attained  inside,  we 
replace  g(y— x)  by  g(y— x)  —  £ \Jy— x  (or  some  similar  function) 
where  £.  is  small.  Then  the  derivative  at  y  *  x  la  — oo,  showing 
that  the  minimum  cannot  be  at  y  ■  x. 

Similarly  in  (2.2),  we  replace  g(y)  by  g(y)  +  £\/y~ and 
h(x-y)  by  h(x~y}+  £\fx~y. 

It  is  easily  seen  that  for  small  £  these  changes  affect 
the  solution  by  a  small  quantity  proportional  to  £,,  and  consequently 
have  no  material  influenoe. 
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